The following content has been provided by the University of Erlangen-Nürnberg.

So, okay, let's start today's lecture. So, welcome everybody to Diagnostic Medicare Image

Processing. Today we will talk about iterative closest point algorithm. So, surely it's mentioned

as ICP. So, here you see the motivation. So, what we want to do is registration of two

different modalities. So, maybe someone can guess what we can see here. Maybe which kind

of modality we can see on top. So, actually it's a male torso. So, we have two different

modalities. Someone wants to guess? Again? It's written there. Okay, good. CT is on top

and TOF is on bottom. So, you don't know which is what, but okay. Okay, on top you see the

CT data. You can see, for example, that you don't have artifacts here, but you have artifacts

here at the side. And this results here in the registration of the two torso. So, you

can see here color encoded the registration result. So, yellow is a good result and red

is a high distance arrow between these two bodies. So, you can see we have here a higher

arrow of 5 millimeters in the red parts, for example, here, which was acquired here in

the TOF data, but not in the CT data. Okay? And on the left side you can see now, for

example, meshes, but you can also think of point clouds or triangles, whatever. So, the

next result you can see here. So, we see here a pork liver. On top again we see CT acquired

data and on bottom it's time of flight acquired data. You can see here that for the time of

flight data we have here acquired something. So, this is the depth information, which results

here if we register both data sets that we get a high arrow here because it wasn't in

the CT data, but in the TOF data. And again you can see here the threat boundaries where

we have this higher arrow than 5 millimeters, which comes here from the time of flight data

and which wasn't acquired with the CT data. Okay. The next result is not medical in this

case, so we have here also this kind of time of flight technology. So, on bottom you see

depth maps. So, we see blue is near to the camera and red maybe is far away from the

camera. And on bottom you see acquired RGB images. And now we want to combine these two

informations. Hello. We want to combine these two informations, so we want to see okay how

far away it is from our camera and we want to have the color information. And what we

do here is we acquire a scene. So, remember you have the camera, you take part of the

scene, then you rotate the camera, you take another part of the scene. And what we want

to do is to reconstruct. So, to register each of the point clouds, meshes, whatever to get

a whole scene of the room. Okay. So, here you can see for example you have two meshes

of any kind of object, for example maybe deliver or whatever. And so we have as an input the

mesh Q and the mesh P. So, Q for example on the right side, P on the left side. And what

we want to get out is the rotation and the translation to come from one mesh to the other

mesh. And you can see here we take the mesh Q, we rotate it and we translate it. Then

we get somehow an estimate Q hat. And this Q hat should be minimal in distance to the

mesh P. Okay. This is what we want to get. So, here is a much more easier example. So,

remember you have two point clouds. And you want now to rotate the blue point cloud, rotate

it and translate it to get the result or that it matches perfectly the red one. So, you

want to minimize the distance between the two point clouds. Okay. So, for the first

part remember we have perfect data. So, perfect data means we have point clouds Q and P for

example. You can see here they have both the same number of points. And you can see here

the green lines are the correspondences between these two point clouds. Sometimes it can happen

of course that the number of points is different as it is here. And you have to get somehow

the point correspondences between these two point clouds. And you can see here for example

a registration result. So, it must not be perfect but nearly perfect. Okay. So, remember

the case we start with the left side and discuss during the lecture how we work if it is not

like this. So, the outline is we first talk about some basics of the ICP. Then the theory.

So, we have parts point to point arrow metric and point to plane arrow metric. These are

the two biggest parts. And then we discuss about some efficient variants of the ICP.